Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add {IoSlice, IoSliceMut}::advance #62987

Merged
merged 1 commit into from
Aug 6, 2019
Merged

Add {IoSlice, IoSliceMut}::advance #62987

merged 1 commit into from
Aug 6, 2019

Conversation

Thomasdezeeuw
Copy link
Contributor

API inspired by the Buf::advance method found in the bytes crate.

Closes #62726.

@rust-highfive
Copy link
Collaborator

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 25, 2019
@alexcrichton
Copy link
Member

Thanks for the PR! As a convenience function though I would imagine that wouldn't this be more useful if defined on something like &[IoVec] or &mut [IoVecMut]? Specifically you typically have a list of buffers and you want to advance them all forward rather than just one at a time which still has its own trickiness in management.

@Thomasdezeeuw
Copy link
Contributor Author

As a convenience function though I would imagine that wouldn't this be more useful if defined on something like &[IoVec] or &mut [IoVecMut]?

I agree, that is what I need for rust-lang/futures-rs#1741 as well. I'm fine with changing it. But where would I put the code for such a change? In the slice file or in IoSlice file?

@alexcrichton
Copy link
Member

Perhaps something like:

impl IoVec<'_> {
    pub fn advance(slice: &mut [IoVec]) { /* ... */ }
}

impl IoVecMut<'_> {
    pub fn advance(slice: &mut [IoVecMut]) { /* ... */ }
}

or something like that?

@Thomasdezeeuw
Copy link
Contributor Author

Looks good, I'll try to update the pr tomorrow.

src/libstd/io/mod.rs Outdated Show resolved Hide resolved
src/libstd/io/mod.rs Outdated Show resolved Hide resolved
src/libstd/io/mod.rs Outdated Show resolved Hide resolved
///
/// # Notes
///
/// Elements in the slice may be modified.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might need expanding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'd recommend going into detail here and explaining how the vector elements of the slice may be modified to adjust the size as noted by advance

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of what I've added?

@Thomasdezeeuw
Copy link
Contributor Author

I've addressed all comments and updated the API to return the truncated slice.

@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-07-31T19:36:16.6033173Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-07-31T19:36:16.6239099Z ##[command]git config gc.auto 0
2019-07-31T19:36:16.6309983Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-07-31T19:36:16.6366412Z ##[command]git config --get-all http.proxy
2019-07-31T19:36:16.6499532Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62987/merge:refs/remotes/pull/62987/merge
---
2019-07-31T19:36:52.9366708Z do so (now or later) by using -b with the checkout command again. Example:
2019-07-31T19:36:52.9366793Z 
2019-07-31T19:36:52.9367063Z   git checkout -b <new-branch-name>
2019-07-31T19:36:52.9367130Z 
2019-07-31T19:36:52.9367208Z HEAD is now at 2a7d79bad Merge 11ea929ea6c0e241253839e6a61dbdf3cbae4953 into e3976fff44e6ce14c2f92252e6a807800b9aa7c0
2019-07-31T19:36:52.9532522Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-07-31T19:36:52.9535492Z ==============================================================================
2019-07-31T19:36:52.9535553Z Task         : Bash
2019-07-31T19:36:52.9535621Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-07-31T19:46:45.5487659Z configure: build.locked-deps    := True
2019-07-31T19:46:45.5487710Z configure: llvm.ccache          := sccache
2019-07-31T19:46:45.5487948Z configure: build.cargo-native-static := True
2019-07-31T19:46:45.5488164Z configure: dist.missing-tools   := True
2019-07-31T19:46:45.5488423Z configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
2019-07-31T19:46:45.5488729Z configure: writing `config.toml` in current directory
2019-07-31T19:46:45.5488774Z configure: 
2019-07-31T19:46:45.5489041Z configure: run `python /checkout/x.py --help`
2019-07-31T19:46:45.5489113Z configure: 
---
2019-07-31T19:48:44.8301446Z     Checking hashbrown v0.4.0
2019-07-31T19:48:48.4768338Z error[E0308]: mismatched types
2019-07-31T19:48:48.4773383Z   --> src/libstd/sys/windows/io.rs:26:27
2019-07-31T19:48:48.4773600Z    |
2019-07-31T19:48:48.4774564Z 26 |         if self.vec.len < n {
2019-07-31T19:48:48.4774852Z    |                           ^ expected u32, found usize
2019-07-31T19:48:48.4775298Z help: you can convert an `usize` to `u32` and panic if the converted value wouldn't fit
2019-07-31T19:48:48.4775509Z    |
2019-07-31T19:48:48.4775750Z 26 |         if self.vec.len < n.try_into().unwrap() {
2019-07-31T19:48:48.4776042Z 
2019-07-31T19:48:48.4783063Z error[E0308]: mismatched types
2019-07-31T19:48:48.4783345Z   --> src/libstd/sys/windows/io.rs:65:27
2019-07-31T19:48:48.4783820Z    |
2019-07-31T19:48:48.4783820Z    |
2019-07-31T19:48:48.4784090Z 65 |         if self.vec.len < n {
2019-07-31T19:48:48.4784562Z    |                           ^ expected u32, found usize
2019-07-31T19:48:48.4784887Z help: you can convert an `usize` to `u32` and panic if the converted value wouldn't fit
2019-07-31T19:48:48.4785103Z    |
2019-07-31T19:48:48.4785534Z 65 |         if self.vec.len < n.try_into().unwrap() {
2019-07-31T19:48:48.4785862Z 
2019-07-31T19:48:48.6382406Z error: aborting due to 2 previous errors
2019-07-31T19:48:48.6382613Z 
2019-07-31T19:48:48.6382959Z For more information about this error, try `rustc --explain E0308`.
2019-07-31T19:48:48.6382959Z For more information about this error, try `rustc --explain E0308`.
2019-07-31T19:48:48.6755472Z error: Could not compile `std`.
2019-07-31T19:48:48.6755572Z 
2019-07-31T19:48:48.6755845Z To learn more, run the command again with --verbose.
2019-07-31T19:48:48.6774793Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "i686-pc-windows-gnu" "-j" "2" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json"
2019-07-31T19:48:48.6784717Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu
2019-07-31T19:48:48.6785007Z Build completed unsuccessfully in 0:00:30
2019-07-31T19:48:48.6785007Z Build completed unsuccessfully in 0:00:30
2019-07-31T19:48:54.7230236Z ##[error]Bash exited with code '1'.
2019-07-31T19:48:54.7263221Z ##[section]Starting: Checkout
2019-07-31T19:48:54.7264815Z ==============================================================================
2019-07-31T19:48:54.7264866Z Task         : Get sources
2019-07-31T19:48:54.7264910Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

///
/// Elements in the slice may be modified.
///
/// # Examples
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation here I think may also want to mention that this API panics if you advance beyond the cumulative size of all buffers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it doesn't panic if you go beyond the cumulative size, see the io_slice_advance_beyond_total_length test.

///
/// # Notes
///
/// Elements in the slice may be modified.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'd recommend going into detail here and explaining how the vector elements of the slice may be modified to adjust the size as noted by advance

/// ][..];
///
/// // Mark 10 bytes as read.
/// bufs = IoSliceMut::advance(mem::replace(&mut bufs, &mut []), 10);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not too fond of this mem::replace dance we have to do.

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-08-01T18:11:52.0284314Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-01T18:11:52.0526461Z ##[command]git config gc.auto 0
2019-08-01T18:11:52.0595708Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-01T18:11:52.0656861Z ##[command]git config --get-all http.proxy
2019-08-01T18:11:52.0789526Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/62987/merge:refs/remotes/pull/62987/merge
---
2019-08-01T18:12:27.4362563Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-01T18:12:27.4362620Z 
2019-08-01T18:12:27.4362996Z   git checkout -b <new-branch-name>
2019-08-01T18:12:27.4363029Z 
2019-08-01T18:12:27.4363071Z HEAD is now at 332b65f80 Merge e16509eedcf741d25439d720e06705169c71d634 into f23a5f208de7fba983bfaaa8d60f1208d5780a98
2019-08-01T18:12:27.4538592Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-01T18:12:27.4541976Z ==============================================================================
2019-08-01T18:12:27.4542037Z Task         : Bash
2019-08-01T18:12:27.4542085Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-01T19:15:32.4481189Z .................................................................................................... 1400/8819
2019-08-01T19:15:38.5941767Z .................................................................................................... 1500/8819
2019-08-01T19:15:51.5730295Z .................................................................i...............i.................. 1600/8819
2019-08-01T19:15:59.2870001Z .................................................................................................... 1700/8819
2019-08-01T19:16:14.7689238Z ...................................................iiiii............................................ 1800/8819
2019-08-01T19:16:26.3001306Z .................................................................................................... 2000/8819
2019-08-01T19:16:28.9478517Z .................................................................................................... 2100/8819
2019-08-01T19:16:32.6994136Z .................................................................................................... 2200/8819
2019-08-01T19:16:40.1824585Z .................................................................................................... 2300/8819
---
2019-08-01T19:20:42.5991620Z .................................................................................................... 5300/8819
2019-08-01T19:20:50.3429647Z ..............i..................................................................................... 5400/8819
2019-08-01T19:20:56.1916427Z .................................................................................................... 5500/8819
2019-08-01T19:21:08.8777960Z .................................................................................................... 5600/8819
2019-08-01T19:21:22.7337305Z ........ii...i..ii...........i...................................................................... 5700/8819
2019-08-01T19:21:39.3627391Z .................................................................................................... 5900/8819
2019-08-01T19:21:44.4079479Z .................................................................................................... 6000/8819
2019-08-01T19:21:44.4079479Z .................................................................................................... 6000/8819
2019-08-01T19:21:58.7565557Z ........i..ii....................................................................................... 6100/8819
2019-08-01T19:22:18.3524782Z ....................................................i............................................... 6300/8819
2019-08-01T19:22:20.5443466Z .................................................................................................... 6400/8819
2019-08-01T19:22:23.0275589Z .....................i.............................................................................. 6500/8819
2019-08-01T19:22:27.6454806Z .................................................................................................... 6600/8819
---
2019-08-01T19:27:16.2198607Z  finished in 23.199
2019-08-01T19:27:16.2394498Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:27:16.4111015Z 
2019-08-01T19:27:16.4112013Z running 146 tests
2019-08-01T19:27:19.8660493Z i....iii......iii..iiii....i............................i..i................i....i.........ii.i.i..i 100/146
2019-08-01T19:27:21.7531716Z iii..............i.........iii.i......ii......
2019-08-01T19:27:21.7533141Z 
2019-08-01T19:27:21.7535566Z  finished in 5.514
2019-08-01T19:27:21.7719708Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:27:21.9370066Z 
---
2019-08-01T19:27:24.0292903Z  finished in 2.257
2019-08-01T19:27:24.0500556Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:27:24.2105460Z 
2019-08-01T19:27:24.2105701Z running 9 tests
2019-08-01T19:27:24.2110111Z iiiiiiiii
2019-08-01T19:27:24.2110524Z 
2019-08-01T19:27:24.2110567Z  finished in 0.160
2019-08-01T19:27:24.2304203Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:27:24.3984760Z 
---
2019-08-01T19:27:43.2580105Z  finished in 19.027
2019-08-01T19:27:43.2777022Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:27:43.4505120Z 
2019-08-01T19:27:43.4505419Z running 122 tests
2019-08-01T19:28:08.3829648Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....i..........iiii..........i...ii...i.......ii.i 100/122
2019-08-01T19:28:13.2059607Z .i.i......iii.i.....ii
2019-08-01T19:28:13.2064564Z 
2019-08-01T19:28:13.2064843Z  finished in 29.928
2019-08-01T19:28:13.2067610Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:28:13.2068708Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-08-01T19:42:52.0590708Z 
2019-08-01T19:42:52.0592028Z    Doc-tests core
2019-08-01T19:42:56.4086402Z 
2019-08-01T19:42:56.4087657Z running 2379 tests
2019-08-01T19:43:09.1945286Z ......iiiii......................................................................................... 100/2379
2019-08-01T19:43:22.1212351Z .........................................................................ii......................... 200/2379
2019-08-01T19:43:52.6935564Z .................................................................................................... 400/2379
2019-08-01T19:43:52.6935564Z .................................................................................................... 400/2379
2019-08-01T19:44:04.1503824Z ..............................i..i.................iiii............................................. 500/2379
2019-08-01T19:44:28.6951745Z .................................................................................................... 700/2379
2019-08-01T19:44:41.3554907Z .................................................................................................... 800/2379
2019-08-01T19:44:53.6850178Z .................................................................................................... 900/2379
2019-08-01T19:45:06.2007599Z .................................................................................................... 1000/2379
---
2019-08-01T19:48:06.7395036Z 
2019-08-01T19:48:06.7443558Z  finished in 0.274
2019-08-01T19:48:06.7462809Z Testing std stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-08-01T19:48:06.9843341Z    Compiling std v0.0.0 (/checkout/src/libstd)
2019-08-01T19:48:27.0863080Z error[E0506]: cannot assign to `bufs` because it is borrowed
2019-08-01T19:48:27.0864298Z     --> src/libstd/io/mod.rs:2705:9
2019-08-01T19:48:27.0864870Z      |
2019-08-01T19:48:27.0865417Z 2705 |         bufs = IoSlice::advance(&mut bufs, 7);
2019-08-01T19:48:27.0866041Z      |         ^^^^^^^^^^^^^^^^^^^^^^^^---------^^^^
2019-08-01T19:48:27.0866664Z      |         |                       |
2019-08-01T19:48:27.0867305Z      |         |                       borrow of `bufs` occurs here
2019-08-01T19:48:27.0867867Z      |         assignment to borrowed `bufs` occurs here
2019-08-01T19:48:27.0868372Z      |         borrow later used here
2019-08-01T19:48:32.1034857Z error: aborting due to previous error
2019-08-01T19:48:32.1035643Z 
2019-08-01T19:48:32.1036222Z For more information about this error, try `rustc --explain E0506`.
2019-08-01T19:48:32.3543157Z error: Could not compile `std`.
2019-08-01T19:48:32.3543157Z error: Could not compile `std`.
2019-08-01T19:48:32.3543247Z 
2019-08-01T19:48:32.3543477Z To learn more, run the command again with --verbose.
2019-08-01T19:48:32.3560279Z 
2019-08-01T19:48:32.3560429Z 
2019-08-01T19:48:32.3561149Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "--target" "x86_64-unknown-linux-gnu" "-j" "2" "--release" "--locked" "--color" "always" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "-p" "std" "--" "--quiet"
2019-08-01T19:48:32.3561297Z 
2019-08-01T19:48:32.3561329Z 
2019-08-01T19:48:32.3574346Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-01T19:48:32.3574444Z Build completed unsuccessfully in 1:29:35
2019-08-01T19:48:32.3574444Z Build completed unsuccessfully in 1:29:35
2019-08-01T19:48:32.8874750Z ##[error]Bash exited with code '1'.
2019-08-01T19:48:32.8914111Z ##[section]Starting: Checkout
2019-08-01T19:48:32.8916069Z ==============================================================================
2019-08-01T19:48:32.8916132Z Task         : Get sources
2019-08-01T19:48:32.8916203Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@Thomasdezeeuw
Copy link
Contributor Author

The commits have become a bit messy, I'll squash them before merging once the review is complete.

@alexcrichton
Copy link
Member

Ok everything looks great to me, thanks! If you want to do some cleanup feel free!

@bors: delegate+

@bors
Copy link
Contributor

bors commented Aug 2, 2019

✌️ @Thomasdezeeuw can now approve this pull request

@Thomasdezeeuw
Copy link
Contributor Author

This is ready now, I've squashed the commits and rebased on master.

@Thomasdezeeuw
Copy link
Contributor Author

Hope this works.

@bors r+

@bors
Copy link
Contributor

bors commented Aug 5, 2019

📌 Commit dad56c3 has been approved by Thomasdezeeuw

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 5, 2019
@bors
Copy link
Contributor

bors commented Aug 5, 2019

⌛ Testing commit dad56c3 with merge ebaa05e81052cc5f04d68880275ba073021d5f9a...

Centril added a commit to Centril/rust that referenced this pull request Aug 5, 2019
…omasdezeeuw

Add {IoSlice, IoSliceMut}::advance

API inspired by the [`Buf::advance`](https://docs.rs/bytes/0.4.12/bytes/trait.Buf.html#tymethod.advance) method found in the [bytes](https://docs.rs/bytes) crate.

Closes rust-lang#62726.
panic!("advancing IoSliceMut beyond its length");
}

unsafe {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we always uphold the minimum standard of having comments on all unsafe { blocks?
I think that's something you should be able to expect of something so central as the standard library.

@Centril
Copy link
Contributor

Centril commented Aug 5, 2019

@bors retry rolled up.

@bors
Copy link
Contributor

bors commented Aug 6, 2019

⌛ Testing commit dad56c3 with merge 1828a3f06b20e9c99e443d2a99fd424016735a05...

@Centril
Copy link
Contributor

Centril commented Aug 6, 2019

@bors retry yielding to r0llup.

@bors
Copy link
Contributor

bors commented Aug 6, 2019

⌛ Testing commit dad56c3 with merge 015346c142811b881b28c5e81c1b27e59fe286d8...

@Centril
Copy link
Contributor

Centril commented Aug 6, 2019

@bors retry yielding to r0llup.

@bors
Copy link
Contributor

bors commented Aug 6, 2019

⌛ Testing commit dad56c3 with merge 766b10a...

bors added a commit that referenced this pull request Aug 6, 2019
Add {IoSlice, IoSliceMut}::advance

API inspired by the [`Buf::advance`](https://docs.rs/bytes/0.4.12/bytes/trait.Buf.html#tymethod.advance) method found in the [bytes](https://docs.rs/bytes) crate.

Closes #62726.
@bors
Copy link
Contributor

bors commented Aug 6, 2019

☀️ Test successful - checks-azure
Approved by: Thomasdezeeuw
Pushing 766b10a to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 6, 2019
@bors bors merged commit dad56c3 into rust-lang:master Aug 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tracking issue for io_slice_advance
6 participants